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We study the quantum anomalous Hall effect described by a class of two-component Haldane 
models on square lattices. We show that the latter can be transformed into a pseudospin triplet 
p + ip-wave paired superfluid. In the long wave length limit, the ground state wave function is 
described by Halperin's (1, 1, —1) state of neutral fermions analogous to the double layer quantum 
Hall effect. The vortex excitations are charge e/2 abelian anyons which carry a neutral Dirac fermion 
zero mode. The superconducting proximity effect induces 'tunneling' between 'layers' which leads 
to topological phase transitions whereby the Dirac fermion zero mode fractionalizes and Majorana 
fermions emerge in the edge states. The charge e/2 vortex excitation carrying a Majorana zero mode 
is a non-abelian anyon. The proximity effect can also drive a conventional insulator into a quantum 
anomalous Hall effect state with a Majorana edge mode and the non-abelian vortex excitations. 



I. INTRODUCTION 

The discovery of the quantum Hall effect (QHE) 1 
opened an era for studying topological quantum phases 2 . 
Some twenty years ago, Haldane 3 - proposed the quan- 
tum anomalous Hall effect (QAHE) for electrons on 
a two-dimensional lattice. This is generalized to the 
time reversal invariant topological insulators 4 in two 
dimensions 5 - - — and three dimensions^— . With the band 
inversion in these system, it raises the hope for realizing 
the QAHE in a two-band model of two-dimensional mag- 
netic insulators^. Candidate materials for this effect, 
HgTe doped with Mni 5 - and a tetradymite semiconduc- 
tors doped with transition metal elements 1 -, have been 
predicted. Other proposals are also made in condensed 
matter systems recently 17 ' 18 . The QAHE has also been 
proposed for cold atom systems^—. 

Another advance is the search for topological phases 
with non-abelian anyons 2 ^ that have potential appli- 
cations for quantum computing—. In addition to the 
v = 5/2 fractional QHE, it has been shown theoretically 
that the superconducting proximity effect on the surface 
state of topological insulator— and on semiconductors 
with strong spin-orbit coupling and Zeeman splitting 25 
provides a new avenue for generating the Majorana zero 
mode and non-abelian vortex excitations. 

In this work, we study the QAHE for a class of two- 
component Haldane models on a square lattice. The 
physical degrees of freedom represented by the compo- 
nents depend on the microscopic details: the real spins of 
electrons, band indices^, the top-bottom surface states 
of three-dimensional topological insulator 1 -, as well as 
the sublattice indices as exemplified below. We show 
that, in a pseudospin representation, the QAHE sys- 
tem can be transformed into a chiral p + zp-wave pair- 
ing state involving both pseudospin components. The 
ground state wave function is given by a determinant of 



the pairing functions whose long wave length limit is a 
charge neutral Halperin (1,1,-1) state analogous to the 
double layer QHE 26 . There are abelian anyon excitations 
with charge e/2, despite that the present model describes 
an integer QAHE system. 



When s-wave superconductor develops in the QAHE 
system due to the proximity effect, tunneling between 
the different isospin components takes place. We find 
that the system displays continuous transitions between 
topological phases with abelian and non-abelian anyon 
excitations. Specifically, a topological phase transition 
from the Hall conductance 2e 2 /h to e 2 /h happens for 
sufficiently strong proximity induced pairing because one 
of the pseudospin components is driven to a strong pair- 
ing state analogous to that in the v = 1/2 double-layer 
fractional QHE discussed by Ho^ 7 - and Read and Greer>2£. 
The remaining pseudospin component is in the weak pair- 
ing state described by a Moore-Read PfaffiaiiS 2 -. A Majo- 
rana zero mode appears in the edge state and the vortex 
excitation carrying this Majorana mode is a non-abelian 
anyon. Interestingly, the model also exhibits a topolog- 
ical trivial phase without the QAHE when the triplet 
p + zp-wave pairing is in a topologically unprotected weak 
pairing state. We find that the proximity effect can drive 
one of the pseudospin components into a topologically 
protected weak pairing state with quantized Hall conduc- 
tance —e 2 /h, edge Majorana zero mode, and non-abelian 
anyon vortex excitations (see also Ref.[29(). 



This paper was organized as follows: 
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II. MODELS 

The Hamiltonian of the two-component model in the 
lattice momentum space is given by 

H = Y2ttPx + i Py") c lk Cbk + h - c - W 

k 

+ h z (k)(cl k c ak - 4 k c 6k ) + /io(k)( C t k c ak + cj k c 6k )] 

where c a ((,\ k annihilate an electron of component a(6) 
with momentum k and ho(k) is the dispersion due to 
hopping among electrons of the same component. The 
physical origin of the terms proportional to p x + ip y and 
h z depends on the system of interest. If (a, 6) label the 
electron spin, p x + ip y — >■ k x + ik y arises from Rashba 
spin-orbit coupling^ and h z is the magnetization. If 
(a, b) are the orbital indices, p x +ip y describes the orbital 
hybridyzatio n 14 ' 15 and h z is the crystal field splitting. 
For the doped tetradymite semiconductors, they consti- 
tute the spin-orbit coupling associated with the three- 
dimensional topological insulator—. 

A. A Generalized Haldane Model in Square Lattice 

In Fig. [T] we give an explicit realization of the Hamil- 
tonian ([lj for spinless fermions on a square lattice where 
(a, b) label the A and B sublattices. This turns out to 
be a modified Haldane's model where the complex hop- 
ping induces ±7r staggered plaquette flux with the link 
phase distribution shown in FigfT] (right panel). The cor- 
responding Halimtonian with this setup is given by 

H o = -^[ c L+5, c ia + ic l a +s y c ^ +h.c] 
ib 

- ^JL^l+s^ia + cl+s^a+h-c] (2) 

ia 

+ fS^\cj i a c-i, + cj ,1 c,. + h.c] 

ib 

i 

where c| b is the creation operator of the electron at the 
site « Q) h on the A or B sublattice. The coordinates and 
vectors are figured in the left panel of Fig. [TJ The lattice 
spacing a is set to be unit, t and t' are the nearest neigh- 
bor hopping amplitude and the next nearest neighbor's. 
±M is the on-site energy on i a and ib, respectively. It 
can also be realized in cold atom context with a simpler 
staggered flux while the atom orbital is different on the 
A and B sublattice^. 

Making the Fourier transformation, the Hamiltonian 
is exactly given by Eq. ([TJ) with p x + ip y — —2t(sink y + 
isink x ), h z (k) — — M — At' cos k x cos k y , and ho = 0, 



where t and t' are the hopping amplitudes between near- 
est and next nearest neighbors indicated in FigfT] and 
±M is the on-site energy for the A and B sublattices. 
Note that the 'magnetization' contains cos k x cos k y and 
is different from that used ini 4 .. A similar model can 
also be realized in cold atom systems 2 -. For ho = 0, 
there is an important 0(2) symmetry associated with 
the U(l) particle number conservation and a Z2 under 
c a k — > icfc,_k, i.e., a C-S (particle-hole • sublattice) 
symmetry. If ho ^ but can be adiabatically driven to 
zero, we think the results of this work are also valid. 

We now study Hamiltonian {JJ in the A-B sublattice 
where k is confined to the reduced first Brillouin zone 
bounded by k x ± k y — ±ir due to the A-B sublattices. 
Our results can be extended directly to other relevant 
cases discussed above with lattice translation symmetry. 

The eigen-energy of ([T]) is given by E — ± \Jh 2 z + \p\ 2 . 
There are two independent Dirac points at (0, 0) and 
(ir, 0) in the reduced zone. When an extended s-wave 
pairing is induced by proximity effect2i on the QAHE 
system, the total Hamiltonian is given by H = Ho + H sc , 
where 

ff 5C = ^/(k)( Co _ kC6k +44_ k ), (3) 
k 

and /(k) = 2A(cosfc a; + cosfc^). The eigen-energy of the 
total Hamiltonian is given by E = ±^J(h z ± /) 2 + \p\ 2 . 
The energy gap of the QAHE at the Dirac point (0, 0) 
closes when A = j (M + At') , whereas the QAHE gap at 
(it, 0) is unperturbed since /(k) = for k x ± k y — ±7r. 




FIG. 1: The square lattice model for Hamiltonian (1) Left: 
The two-sublattice and hopping. Right: The flux distribu- 
tion. Hopping along arrowed vertical links generates a phase 
7r and arrowed horizontal links a phase 7r/2. A net flux of 
— 27r (7r/2) is accumulated for the dark (light) grey triangular- 
blocks. The rest of the hopping is real. 

To unveil the ground state wavefunction and the topo- 
logical properties of Eq. ([1]) , we introduce a pseudospin 
representation by mixing the electron and hole of differ- 
ent component (which in fact indicates a particle-hole 
transformation of 6-component) 

&t- k = (c ak + 4_ k )/ V2, cj. k = i(c ak - c\_ k )/V2. (4) 
Under this unitary transformation, the Hamiltonian in 
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terms of fermions carrying the pseudospin becomes 
H Q = ^ [hzcl k c sk -((p x -ipy)c s uc s - k + h.c.)/2].(5) 

k;«=t4 

Both the pseudospin- j" and J, fermions, having a band 
dispersion h z (h), are in the p + ip-wave paired states. 
Hamiltonian is closely related to the v = 1/2 double- 
layer fractional QHE if we identify the pseudospins with 
the even/odd states of the isospin (layer index) in the 
context of triplet chiral p-wave pairin g 27 i 28 . 

B. Winding Numbers 

We now calculate the winding number, which describes 
the mapping from the reduced zone (a torus) to a target 
sphere specified by the unit vector n = ^ Px ' p ^ ,hz ' . The 
winding number is given by C = C-j- + Cj, with C s — 

J d 2 kn ■ 9fe x ii x 9fc y n in the continuum limit. A direct 
calculation yields = Cj, = 1 and C = 2 which reflects 
the fact that the whole first Brillouin zone covers twice of 
the sphere with such a map n. This result can be under- 
stood intuitively by considering the vector components of 
n near the two Dirac points (0, 0) and (tt, 0). For a small 
deviation q ~ 0, they are (—2tq y ,2tq x ,—M — At')/E 
near (0,0) and (-2tq y , -2q x , —M +4t')/E near (tt,0) 
(See Fig. [2]). Therefore, the Dirac points are mapped 
to the north and south poles which are covered once. 
A semi-sphere including a pole contributes ±1/2 to the 
winding number depending on the pole's frame. When 
M < At', both poles are in the right hand frame and 
each semi-sphere contributes 1/2 which gives = 1 

2 

and C = 2. Thus, this QAHE has Hall conductance 2^-. 

(a) (b) 

FIG. 2: : The frames and frame change from M < At' to 
M > At', (a) (~2tq y ,2tq x , —M - At') at (0,0). The frame is 
not changed and is the right, (b) (—2tq y , —2q x , —M + At') at 
(7r,0). The frame is changed from the right for M < At' to 
the left for M > At'. 



III. GROUND STATES AND ANYONIC 
EXCITATIONS 

A. The (l,l,-l)-state and abelian anyons 

The paired state has BCS coherence factors |M s ,k| 2 = 
|(1 + -gj), |v s ,k| 2 = 5(1 — w) an d pairing functions 

g s (k) = u S) k/u s ,k = .g(k) = -(-Eq - h z )/p. (6) 



For M < 4i', the effective chemical potential at the Dirac 
point (7r,0) H( n .o) — — ^ z ,(7r.o) = M — At' < such 
that u s ~ 1 and g s ~ v s ~ q y + iq x . This is in the 
strong pairing regime 2 - and can be thought as the 'infin- 
ity' point in the continuum theory. On the other hand, 
the effective chemical potential at the Dirac point (0, 0) 
M(o,o) = — hz-,(o,o) — M + At' > which leads to v s ~ 1 
and g s ~ l/u s ~ l/(q y — iq x ). This singular pairing 
function in the long wavelength limit is the hallmark of 
the topologically nontrivial p + ip weak pairing phase 28 . 
Therefore, the long distance, low energy physics is deter- 
mined by the weak pairing of the qusiparticles carrying 
both pseudospins near the Dirac point (0, 0). The ground 
state is given by 

\G S ) ex exp[i ]T 9(k)(c\ k c\_ k + c{ k 4_ k )]|0 s ), (7) 

k 

where |0 S ) is the vacuum for the fermions carrying the 
pseudospin, c s k|0 s ) = 0. From the transformation (U), it 
is clear that this vacuum is empty of the a-electrons but 
filled with the ^-electrons. The ground state in Eq. ([7]) 
resembles the neutral part of the (3, 3, l)-state in v — 
1/2 double layer fractional QH E 26 i 28 . Going back to the 
original a-b component electrons, the ground state wave 
function of Eq. ([!} is of the form of a determinant of the 
pairing function g(r), the Fourier image of g(k), 

*(K,ii}) = (0|n(*,(r?)c6(rJ)|G) cx det[ 5 (r« - r))], 

i 

where r®' b are the coordinates of the two-component 
electrons in the ground state \G) out of the vacuum 
c a(b)fc|0) = 0. The ground state \G) and the vacuum |0) 
of the original Hamiltonian ([TJ are related to \G S ) and 
|0 S ) in the pseudospin representation by \G S ) = U\G), 
\0 S ) = U\0) where U is the unitary operator for the 
particle-hole transformation in the 6-component indi- 
cated in Eq. In the long wavelength limit, the weak 
pairing near (0,0) implies g(rf — rj) ~ z ^\ z b ■ Using 
Cauchy's determinant identity, we find 

r b i})~n(z? 4) JIM - z\) - z b r y\ 

i<j k<l sr 

This is exactly the charge neutral part of the (3,3,1), i.e. 
a (1, 1, — l)-state. 

The above result allows us to describe localized charge 
excitations in the filled bands which are carried by the 
finite k states. A Laughlin-type quasihole excitation is a 
good approximation of a real hole with charge e because 
the number of extended states in the QAHE system is 
the same as that in a band insulator. Thus, the minimal 
charge e/2 vortex pair excitations located at rji and 772 
are similar to that in the (3,3,1) state, i.e., 

Mm,m) - det{[(z-- m )(z b J - m ) + (l o 2)M<-r*)}. 

We thus come to the conclusion that the vortex excita- 
tions in this integer QAHE system are charge e/2 abelian 



anyons carrying a neutral Dirac fermion zero mode. This 
fractional charge can also be understood from the ground 
state degeneracy. In Eq. ([5]), the 0(2) symmetry is gen- 
erated by the pseudospin S y = —i X)k( c }k c ik — c |k c tk)/2 
and c^k ^ c^k- Thus, the ground state is four-fold de- 
generate and the vortex excitations carry S y — 1/ 42&. In 
terms of the original QAHE system, 2S y corresponds to 
the charge operator N a + Nf, due to the partial particle- 
hole transformation. Thus, the vortex carries charge e/2, 
which is different from e/4 in the (3,3,l)-state. The 
vortex pairs have an excitation energy on the order of 
e 2 /(4|?7i — 772!) and are thus stable when the vortices are 
well separated by the distance I > e 2 /[8(M +4±')]. When 
the vortices are close enough so that their energy is higher 
than the band gap, they fuse into a real hole. 



LU 



LU 




B. Weak-strong pairing phase transition and 
non-abelian anyons 

In the pseudospin representation, the pairing due to 
the proximity effect in Eq. ([3]) plays the role of a mag- 
netic field and h z in Eq. ([5]) splits into /i Z ;t,4- — T /(k) 
for Cf^, respectively. In the double-layer language, 
this /(k) amounts to inter layer tunneling that splits 
the symmetric and antisymmetric combinations. The 
antisymmetric component can be driven into a strong 
pairing state, leaving only the symmetric one govern- 
ing the physics at low energy and long wavelength 27 . 
The winding number associated with the pseudospin- 
up (-down) component is defined by the unit vector 
n t(4) = (Px,P y ,h z . t{i)M )/E. Since /(k) = at (tt,0), 
n-^j, = n at this Dirac point. Near the Dirac point (0, 0), 
n t 4 = (-2tq y ,2tq x ,-M - it' =p 4A)/J5. For M < At', 
the energy gap at (0, 0) closes at A = A = \(M + At'), 
When A > Ao, at (0, 0) changes from the right frame 
to the left frame while n-j- remains in the right frame. 
Therefore, Cf = 1 while the pseudospin-down compo- 
nent becomes topological trivial with C4. = 0. This can 
also be seen from the effective chemical potential of the 
components; only /XfYo.o) = — ^z,t;(o,o) > satisfies the 
weak pairing condition, while the other three are in the 
strong pairing regime, i.e. Mi;(o,o) = -^,4. ; (o,o) < and 
Mt,4;(7r,o) = - h z,tM*,o) < 0. Thus, the ground state 
wave function is given by *({r",rj}) ~ Pf[</-f(rj — r^)], 
which is identical to the charge neutral Moore- Read Pfaf- 
fian in the long wavelength limit. The ground state is 
three-fold degenerate^. A pair of vortices are the same 
as the Moore-Read non-abelian anyons 2 ^ 

Vvivi, m) ~ Pf{[(*< - m)(zj - m) + (l ^ 2)]^ - r,)}. 

However, each vortex again carries charge e/2. The safe 
distance between the vortices is also determined by the 
band gap. 



FIG. 3: (color online) The energy dispersion E along k x 
showing the evolution of the edge states with proximity in- 
duced pairing A. Top panel starts with (la) the QAHE state 
{M < At') while the bottom panel starts with (lb) the in- 
sulating state (M > At') when A = 0. The gap closes at 
A = Ao (lb and 2b). The chiral edge states located at the 
y = and y — L boundaries are marked by red dashed lines 
and, in the case of degeneracy, the blue solid lines, t = At'. 



C. Insulator-QAHE transition and non-abelian 
anyons 

Interestingly, for M > At', the model Hq with the prox- 
imity effect is topologically trivial with = 0, since 
the poles at (%, 0) and (0,0) are in the opposite frames 
(See Fig. HJb)). In this case v s ~ 1, but u s ~ q y — iq x 
near (0, 0) while u s ~ q y + iq x near (jr, 0). As a result, the 
quasiparticle local wave functions near the Dirac points 
are the holomorphic (1, 1, — l)-state for (0,0) and the 
anti-holomorphic (1, 1, — l)-state for (7r, 0). They con- 
tribute to the Hall conductance with the same magnitude 
but opposite signs and lead to an insulating state. 

In the presence of proximity induced pairing, the in- 
sulating gap closes at A = A . When A > A , the 
pseudospin-up component is pushed into the strong pair- 
ing regime near (0,0), i.e. Hf^o.o) < 0; while all others 
remain unchanged. Thus the low energy physics is dom- 
inated by the Pfaffian of the up-spin component near 
(tt, 0) with winding numbers Cf = — 1 and = 0. 
This is therefore a transition from an insulator to a non- 
abelian QAHE state with Hall conductance —e 2 /h where 
vortex excitations are non-abelian anyons 2 ^. 



IV. EDGE STATES AND MAJORANA 
FERMION MODES 

Next, we present explicit calculations of the edge state 
spectrum of the total Hamiltonian H under an open 
boundary condition along the y-direction of a strip. The 
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existence of the nontrivial topological invariants implies 
stable gapless edge modes separated from the gapped 
bulk excitations. For a small M < At' , say M = 0, 
gapless chiral edge states appear across k x = as shown 
in Fig. [3] There are indeed two degenerate chiral edge 
states in this (1, 1, —1) abelian QAHE state (Fig.[3[la)). 
The gap closing is demonstrated in Fig. EH lb). After 
the proximity induced topological phase transition, a sin- 
gle edge mode survives (Fig. EJlc)) in the non-abelian 
QAHE state. The lower panel of Fig. [3] starts with the 
non-topological insulator at M = AM' where edge states 
are absent. The proximity effect causes the topological 
gap closing transition (Fig.[3][2b)) and the emergence of a 
single edge state in non-abelian QAHE state (Fig.|3[2c)). 



results obtained in previous sections. For k x — + q x <C 
1, the gapless edge modes can be written as ip s {0] q x ) — 
u s (0;q x )c s>0;qx + v s (0;q x )cl ._ gx . At q x = 0, we have 
u s (q x ) = v(q x ) = A= and ipt(0;0) = rp s (0;0). These are 
indeed the Majorana fermion zero modes. In the (1,1,-1)- 
state, both ip-\-,i exist and combine into a complex neutral 
Dirac fermion, which is consistent with the neutrality of 
the bulk ground state in the long wavelength limit. For 
both the holomorphic Moore-Read Pfaffian (Fig. EIlc)) 
and the anti-holomorphic Pfafhan states (Fig. El2c)) in- 
duced by the proximity effect, there is only one Majorana 
mode on the edge, consistent with the existence of the 
non-abelian anyons. 



A/Ao 
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MR Pfaffian 
(Non-Abelian) 

C=1-QAH 

Majorana fermion 



(1,1,-1)Abelian 
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Dirac fermion 
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MR Pfaffian 
(Non-Abelian) 

C=-1-QAH 

Majorana fermion 



Band Inslator 
C=0 

No edge state 



M/4t' 



FIG. 4: The phase diagram with the Chern numbers, the 
ground state wave functions and the gapless edge modes. 

These numerical calculations confirm the analysis and 



V. PHASE DIAGRAM AND CONCLUSIONS 

In Fig. 2J we summarize our results in a schematic 
phase diagram for this class of two-component abelian 
and nonabelian QAHE. While the interesting non-abelian 
phase was confirmed after a superconductor proximity, 
the C ■ S discrete symmetry leads to a new topological 
order with abelian anyon even without a superconduc- 
tor proximity and no interaction between electrons are 
concerned. Finally, we argued that the partial particle- 
hole transformation may be a general duality between a 
topological insulator and a topological superconductor. 
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